Zero-shot domain paraphrase with unaligned pre-trained language models

Abstract Automatic paraphrase generation is an essential task of natural language processing. However, due to the scarcity corpus in many languages, Chinese, for example, generating high-quality paraphrases these languages still challenging. Especially domain paraphrasing, it even more difficult obtain in-domain sentence pairs. In this paper, we propose a novel approach domain-specific zero-shot fashion. Our based on sequence-to-sequence architecture. The encoder uses pre-trained multilingual autoencoder model, and decoder monolingual autoregressive model. Because two models are separately, they have different representations same token. Thus, call them unaligned models. We train model with English-to-Chinese machine translation corpus. Then, by inputting Chinese into could surprisingly generate fluent diverse paraphrases. Since inconsistent understandings language, believe that paraphrasing actually performed Chinese-to-Chinese manner. addition, collect small-scale computer science. By fine-tuning corpus, our shows excellent capability domain-paraphrasing. Experiment results show significantly outperforms previous baselines regarding Relevance, Fluency, Diversity.